All Questions
Tagged with databasedata-structures
64 questions
2votes
0answers
71views
Efficient FIFO Data Storage and Deletion for Sequential Data [closed]
I’m building a device that needs to store data sequentially, retrieve it in chunks using FIFO, and then delete it. My device uses the ESP32-S3, and I'm currently using an SQLite build. However, as the ...
-1votes
2answers
316views
Text files vs. MySQL Performance for Large Dataset
I am currently making a system for users to generate flashcards for Languages. This involves adding information such as the definition, pronunciation, and example sentences for a word. So far, I have ...
0votes
1answer
195views
How is B-Tree used in DBMS-s if WHERE condition can be anything?
The most of the database management systems use B-tree as a data structure for increasing the performance. Let's imagine that we have a table users with the following columns: id (int), name (string), ...
- 111
0votes
1answer
110views
Data Class: spread boilerplate for each different database over multiple class copies, or add complexity to original class?
I'm creating a prototype mobile app that will store data locally in an sqLite database and also on the cloud (CloudKit for now). The cloud storage uses a certain way of storing the data, and sqlite ...
- 1,660
1vote
3answers
605views
What makes a data pipeline scalable? Best practices for scalable design?
I have been searching about this topic for a few days and have not yet found anything on books, courses or tutorials. What is a way to make data pipelines more scalable, that doesn't involve NoSql or ...
-1votes
1answer
573views
Best way to store and fetch image bytes?
I have a case in my startup where I need to fetch images in bytes as fast as possible. At the moment I'm storing the images in Azure Storage then I fetch them on the run and cache them into my ...
- 107
2votes
1answer
227views
Implementing a persistent searchable "queue"
I'm receiving a lot of a data entries from different (IOT) devices. I need to temporarily and persistently store this data until it is polled by the matching clients, after which it will be deleted. ...
- 29
1vote
1answer
324views
Data structures - prioritizing records based on pending actions
I need to add a priority column to an orders database to sort them, which will vary depending on what has already been done (actions taken). The other catch is: there are two different roles that will ...
- 127
2votes
2answers
83views
Make modifications to a database resource specific per company
Context I've been searching the internet to see if I could find any other information or posts about what I'm trying to achieve, but I had no luck. I don't know if there's a specific term or whatever ...
- 43
0votes
1answer
335views
Does it make sense to have redundant DB fields to improve query performance, programming speed and code readability?
I am a bit torn about the database design for a solution. I have the following structure: (only the relevant fields are marked with #) +------------------+ | activities | |------------------| | ...
- 129
2votes
2answers
256views
API architecture design for fast reads of text file with 150m unique labels
Assume a text file with 150m unique records. Each record has two columns: (1) string and (2) integer. The string is a unique label, and the integer is the label's value. The only query will return the ...
- 243
0votes
1answer
73views
Organizing Data with Specific Group in Objects and Lists
So recently I have been working with a large-scale app that has a large number of individual objects with specific group attributes. For example, I have a list with objects such as {Name: "Todd", ...
- 23
1vote
1answer
155views
Storing, indexing and searching data using Azure with frequently updated fields. Multiple Sources
We have a multi-vendor marketplace (e-Commerce) system and plan to move our data structures into a polyglot architecture to improve the performance of reads (critical) and writes (not as critical). ...
2votes
5answers
2kviews
How do B-Trees used in databases sort data based on different fields?
I know majority of databases uses B-Trees, and I can see how using a balanced binary tree will give fast sort times, for ordering by ID or whatever else the primary key is; but how are databases able ...
2votes
2answers
3kviews
Is it allowed to calculate the derived attributes by using other tables' attributes directly?
I tried to build a database for Driver Booking system. Here's brief details: Drivers have several vehicles and they offers the driving service with them Drivers will charge different hourly rates on ...
- 31
